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Abstract. We establish two moderate deviation principles (MDP) in the 
bootstrap setting. We prove MDP for the joint distribution of the empirical 
measure and the empirical bootstrap measure (empirical measure obtaining by 
the bootstrap procedure). We derive MDP for the conditional distribution of 
the empirical bootstrap measure given the empirical probability measure. For 
most common statistical functionals (in particular diffcrcntiablc and homo- 
geneous functionals) we show that their asymptotics of moderate deviation 
probabilities in the cases of empirical measure and bootstrap empirical boot- 
strap measure coincides. However the moderate deviation zones are different. 



1. Introduction. 

Let 

- S be a Hausdorff topological space; 

- & the cr-field of Borel sets in S; 

- A the space of all probability measures on (S, 

Let X±, . . . ,X n be i.i.d. random variables, taking values in S and P(£ A) be 
unknown distribution of X\ . 

Denote P n the empirical measure (occupation measure) for X±, . . . ,X n , that is, 
for any ^"-measurable set A, 

n 

P n {A) = -Y / I(X i eA) 
t=i 

In 1979, in a landmark paper Efron [T2] proposed to analyze the distributions of 
statistics V(X\, . . . , X n ) with the help of the bootstrap procedure. In the bootstrap 
procedure we consider the empirical measure P n as an estimator of the probability 
measure (pm) P and simulate the distribution of statistics V{X\, . . . ,X n ) on the 
base of pm P n . In other words, we simulate independent copies {X^ il . . . , -X^iJiefijk] 
of i.i.d random variables such that X* x is distributed according to P n . After that 
the empirical distribution of (V^X*^ . . . , A^* i )) ie [ 1 fe j is postulated as an estimate 
of the distribution of V(X\, . . . , X n ). 

It is of interest to estimate large and moderate deviation probabilities of V (Xi , . . . , X, 
Such problems emerge constantly in confidence estimation and hypothesis testing. 
The significant levels in the confidence estimation and the type I error probabilities 
in hypothesis testing are (usually) of small values and thus are compatible with 
LDP - MDP analysis. Hence it appears natural to compare V(Xi, . . . , X n ) and 
V{X*, . . . , X*) in terms of LDP - MDP approach. 

In this paper we carry out such an MDP based comparison in the following setup. 
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We represent V{Xi, ... , X n ) and V{X{, ... , X*) as functionals of P n and P*, 
where P* is the empirical probability measure of X*, . . . , X*, i.e. 

V(X 1 ,...,X n )=T(P n ), 

v(x* u ...,x*) = t(p:) 

Thus wc reduce the problem to an MDP study for T(P*)-T(P n ) and T(P n )—T(P). 

The LDP - MDP analysis for empirical measures generated i.i.d. random ob- 
jects is well known from Sanov |24j , Grocneboom, Oostcrhoff and Ruymgaart |17j , 
Borovkov and Mogulskii [5] , Dembo and Zeitouni [10] , Eichelsbacher and Schmock 
[13], Arcones [2], de Acosta [5], Ermakov [T5] (see also references therein). The 
results there are obtained under rather general assumptions. 

Our goal is twofold. 

1. We develop MDP technique from the above mentioned papers for 

(p: -p n )x (p n - p). 

and implement the above result for the MDP comparison of 
T(P n ) - T{P) and T{P*) - T(P n ). 

2. We establish the MDP for a conditional distribution of the empirical bootstrap 
measure P* given empirical probability measure P n . 

We notice that the MDP for the joint 

"empirical bootstrap + empirical probability" 

measures is valid in a "smaller time zone" than the MDP for empirical measure 
only. On the other hand, the time zone for the above-mentioned conditional MDP 
is essentially larger with probability close to one. The first statement shows insta- 
bility of a bootstrap procedure provided that the empirical measure belongs to the 
MDP zone. The second statement confirms the wellknown fact that the bootstrap 
statistics have more stable properties (see Hall [T5], Wood [28], DasGupta [3]). 

The LDP for the empirical bootstrap measure has been studied in Chaganty [7] 
and Chaganty, Karandikar [5] using weak convergence. In contrast to that, for the 
MDP analysis we use T$-topology (see, Arcones [2]) enabling treatment of statistics 
having unbounded influence functions. 

Due to involved structure of the rate function, the LDP result for P* x P n is far 
from being "applicable" even for simple statistical cases (as exceptions, see special 
cases in Chaganty [7]). In contrast to that, the MDP provides readily derived 
asymptotics which are compatible with a majority of widespread statistics and 
thus the MDP effectively serves T(P n ) - T(P) and T(P*) - T(P„). In particular 
we show that the asymptotics of moderate deviation probabilities of T(P n ) — T(P) 
and T(P*) — T(P n ) are calculated easily for the statistical functionals having the 
Hadamard derivatives. 

The assumption of differentiability is the standard tool for the proof of asymp- 
totic normality of statistics T(P n ) (see Serfling [22], van der Vaart and Wellner [2"T] 
) and, in implicit form, were also used for the study of moderate deviation proba- 
bilities (see Aleskeviciene pQ, Jureckova, Kallenberg and Veraverbeke [3D]; Inglot, 
Kallenberg and Ledwina (TpJ, Arcones [3]). The moderate deviations of statistics 
were studied in Ermakov [TS] for the case of Freshet derivative and Gao and Zhao 
[IB] for the case of Hadamard derivatives. In [TB] the statistical functionals were 
considered as functionals of empirical distribution functions and the technique of 
large deviations of stochastic processes was implemented. We consider the statis- 
tical functionals as the functionals of empirical probability measures or empirical 
bootstrap measure. 
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The remainder of the paper is organized as follows. In section 2 we present 
MDP for empirical bootstrap measure, empirical measure and conditional MDP 
for empirical bootstrap measure given empirical measure. The moderate deviation 
probabilities of statistical functionals are studied in section 3. The proofs of MDP 
and conditional MDP are given in sections 4 and 5 respectively. 

2. MDP for empirical and empirical bootstrap measures 

2.1. Notations. Throughout the paper, the following notations are implemented: 

- Q2 x Qi the Cartesian product of probability measures Q2, Qi G A; 

- A 2 = A x A denotes the set of all measures Q2 x Q\ with Q2, Qi € A; 

- C, c are generic positive constants; 

- x(^) is the indicator of event A; 

- [t] is the integral part of real number t; 

- J always denotes J s . 

2.2. -r#-topology. We begin with the definition. 

Let us fix a decreasing sequence of positive numbers (& n )n>i with properties: 

b n -> 1 

nbl -> 00 \ n -> 00. (2.1) 
>- -> 1 

Denote $ the set of measurable functions / : S — >• R with the following property: 
lirn -L l g(nP(|/(X)| > 6,; 1 )) = -00. (2.2) 

Let 

A$ = |p G A : y |/(X)|dP < 00, forallf G $}. 
The coarsest topology in A$ providing the continuous mapping 



/ dQ, for all / G $, Q G A$ 

is known as the r$-topology (henceforth, all topological concepts refer to the r$- 
topology). Denote 0$ the smallest er-ficld that makes all these mapping measurable. 

For any set fi C A$ the notations: clo(fi) and int(fi) are used for the closure 
and interior of Q respectively 

The r$-topology in A| is the corresponding product topology. For the set $o(C 
<f>) of all real bounded measurable functions, the r$-topology coincides with the 
r-topology (see GOR [T7], Dembo and Zeitouni [TU], Eichelsbacher and Schmock 
[IB] ). For P, Q £ A and P, Q G A$ we define the sets A and A $ respectively of of 
all differences P — Q. The r$-topologies in Ao$ and Aq$ are defined in a standard 
way as well as clo(fio) an d int(fio) the closure and interior of Oq C Aq $ . 



2.3. Rate functions. For G G Ao, let 

fi/(f) 2 dP, G«P 
00, otherwise 



Po(G|P) 



be the rate function (in statistical terms, 2(%(G\P) is the Fisher information) which 
arises naturally in the MDP analysis of empirical measures P n (see Borovkov and 
Mogulskii [S]; Gao and Zhao [TB], Arcones [2] and Ermakov [TS] ). 

In the bootstrap setting, a rate function (we shall denote it by p^ b ) is constituted 
from two ones: 

p 2 0b (G\P) = P l(G 2 \P) + pUGi\P), 
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where G — G 2 x G\ £ Ag $ 

2.4. MDP for empirical bootstrap measure. For any set A £ J£" and any 

signed measure G £ A denote |G|(A) = sup{G(P) - G(D) : B QA,D QA}. The 
measure \G\ is the variation of signed measure G. 

Let the signed measures if, H n € Ao$ satisfy the following assumptions. 

A. We have P n = P + b n H n £ A$, P + b n H G A$ and H n — > H as n — > oo in the 
T$-topology. 

Bl. For any / G $ 

limsupsup(n^)" 1 log I nb n / x(|/(a;)| > ^n 1 ) d\H m \ J = -oo. 

Define the signed measure O £ Ao* such that O(A) = for all measurable sets 
Ae3. For each G £ A $ denote G = O x G. 

Theorem 2.1. Assume A and Bl. Let £Iq C Aq $ 6e er$ measurable set of Aq $ . 
T/ien f/ie Moderate Deviation Principle (MDP) holds 

liminf(n^)- 1 log P„((P*-.P„) x (P«-P ) G ^o) > -^ 6 (int(no -■&),-?) (2-3) 

and 

limsup^)-! l og P„((P*-P„) X (P n -P) G &»Oo) < -/j§ t (cl(fi -#),P). (2.4) 

n— > oo 

Remark 2.1. In hypothesis testing, the type II error probabilities are often ana- 
lyzed for the alternatives P„ converging to the hypothesis P. Theorem 1 2 . 1 1 allows 
to study moderate deviation probabilities in this setup. The analysis of importance 
sampling efficiency is also based on MDP with a sequence pm's P n converging to 
pm P(see Ermakov [15]), Naturally, if we suppose that H n ,H are absent, we get 
the usual form of MDP. Bolthausen [4] has proved the Donsker-Varadhan LDP [11] 
when the laws of random variables converge weakly and a uniform exponential inte- 
gration condition is satisfied. Theorem 12 . 1 1 and further Theorems can be considered 
as versions of these results. 

The modern form of LDP-MDP (see de Acosta [5] , Gao and Zhao (TB] , Leonard 
and Najim [21] ) covers the case of unmeasurable sets SIq and is given in terms 
of outer and inner probabilities. Let (T,!J, P) be a probability space. The outer 
probability of an arbitrary subset B C T is 

(P)*(P) = M{P(A);B C A,A£ a Aoe J 

and (P)*(P) = 1 - (P)*(A e,, \ B) is the inner probability. All Theorems of the 
paper hold also for this setup. In Theorem l2.1l it suffices to replace pm's P n in (|2.3[) 
with (P n )* and pm's P n in (]2.4[) with (P n )«. 

The bootstrap procedures are often implemented with sample size k =/= n. In The- 
orem [22] given below the results are extended to this setting. Let X±, . . . ,X% be 
i.i.d.r.v.'s having pm P„. Denote P^ the empirical measure of X £ XJ1 . Suppose 
that k = k(n), k/n — > ^ > as ?i — > oo. 

For any G = G2 X G\ £ Aq define the rate function 

pl(G: P)=upl(G 2 :P) + pl{G 1 : P). 

For any Cl C A§ denote p 0u (£lo : P) = inf{ p 0u (G : P) : G £ fl }■ 

Theorem 2.2. Assume A and Bl. Then the Moderate Deviation Principle (MDP) 
holds 



liminf(n62)-ilogP„((P fc *-P„)x(P n -P ) £ b n Cl ) > - p 2 0u (int{n - H) , P) (2.5) 

n— J-oo 
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and 

]imsup(nbl)- 1 \ogP n ((PZ-P n )x{P n -P)£b n n ) < -p% v {d(n -H),P). (2.6) 

n— >oo 

The proof of Theorem l2.2l is akin to that of Theorem 12 . 1 1 and is omitted. From now 
on, we assume k — n. 

2.5. MDP for empirical measure. In section a version of Arcones [2] MDP for 
empirical processes is given in the setup for empirical pm. These results were es- 
tablished in Ermakov [T3] and are given here for the comparison with the bootstrap 
setup. 

The MDP for the empirical probability measures holds for wider zones of mod- 
erate deviations. Define the set Vl 7 of measurable functions / : S — > R 1 such that 

lim (ndlr 1 \og(nP(\f(X)\ > nd n )) = -co (2.7) 



n— foo 

;2 



where d„ — > 0, nd n — > oo, d n+ i/d n — > 1 as n — > oo. 
Assume the following. 
B2. For any / e * 



lim (nd?) 1 sup log ( nd n / 

n->oo m \ J 



X (\f{x)\ > nd n ) d\H m \ ) =-oo. (2.8) 



Using the reasoning of Lemma 2.5 in Eichelsbacher and Lowe [12], we get that Bl 
or B2 implies 



sup m J f z d\H m \ < oo (2.9) 

and ([2~2jl or ([27T|) implies 

' fdP < oo. (2.10) 



In Lemma 2.5 in [Tl] (|2.10[) has been proved, if d n is decreasing and ri-l 2 d n is 
increasing. Since d n /d n -i — > 1 as n — > oo we can choose a subsequence d nk such 

1/2 

that n k d nk is increasing and d rik / d„ k _ l — > 1 as k — > oo. After that we can choose 

a subsequence d„ k , such that d njs .. is decreasing and dn k ./d nk ._ 1 — > 1 as i — > oo. 

Implementing to the subsequence the same reasoning as in the proof of Lemma 
2.5 in [H] we get ([2~TUj) . 

Theorem 2.3. issume j4 wii/i $ = \I/ and B2. Let D,q be a^, measurable set of 
A * . Then the MDP holds 

lhnM(nd 2 n )- x logP„(P„ E P + d n n ) > -^(int(O - H), P ) (2.11) 

n— >oo 

and 

limsup(nd 2 t )- 1 logF n (P„ £P + 40o) < -pg(c[(O - ff), P ) (2.12) 



The Theorem 12.41 given below shows that the MDP for the empirical bootstrap 
measure can not be valid if (|2.2[) is replaced by (|2.7[) . 

Theorem 2.4. Lei t/ie random variable Y = |/(A)| satisfy i2.2\) . Let the sequences 
r n and e„ be such that b^ 1 < r n ,b~ e n — > oo, ne n /r n — > oo as n — > oo and 

lim (nel)- 1 log (nP (Y > r„)) = 0, (2.13) 
lim(r„e„)- 1 log^=0. (2.14) 
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Let Y\, . . . ,Y n be independent copies of Y and let Y* , . . . , Y* be obtained from 
Y\ , . . . , Y n using the bootstrap procedure. Then 

lim {nelyHogP I V Y* > ne n ] = 0. (2.15) 

\4=1 / 

The proof of Theorem 12.51 is given in Appendix. 
Example. Let P(Y > t) = exp{-f},0 < 7 < 1. Then b„ = o(n~^). By 
straightforward calculations we get that (|2.13|) . (|2.14[) hold for any sequence r„ = 

ri^i f n , e„ = n~2T7 f£~ S where (logn.) 1+ 'i~ s << /„ << n (2+-rxi+«> anc l < 
<5 < -j. Thus the moderate deviation zone in Theorem 12.11 can not be improved 
essentially for such an asymptotic of P(Y > t). 

2.6. MDP for conditional bootstrap measure. Thcorcm l2 . 5 1 given below shows 
that the MDP holds almost sure (a.s.) for the conditional distribution of the empiri- 
cal bootstrap measure given the empirical probability measure. We call this version 
of MDP the conditional MDP. In this model we allow the sample size k = k„ of 
bootstrap procedures to have values different from n. 

Almost sure version of conditional LDP for the bootstraped sample mean has 
been established Li, Rosalski and Al-Mutairi [22]. Chaganty and Karandikar [6] 
have proved conditional LDP for empirical bootstrap measure in the case of weak 
topology. 

For a sequence of arbitrary random variables Z n : S — > R 1 (Z n may not be 
Borel measurable) we say that lim inf n^oo Z n > c inner almost surely (a.s* if 
there exist a sequence A„ of measurable random variables A„ < Z n such that 
P(liminf„_ >00 A„ > c) = 1. 

We say that lim sup,^^ Z n < C outer almost surely (a.s* .) if lim infn^oo — Z n > 
— c a.s*. 

We say that limsup,^^ Z n = —00 outer almost surely (a.s*.) if lim inf n^oa —Z n > 
— c a.s* for any C > 0. 

Let . . . , X£ be i.i.d.r.v.'s having pm P n . Denote P£ the empirical proba- 
bility measure of , . . . , X% . Suppose that ^ < c < 00 and k n — > 00 as n — > 00. 

For each t > 2 define the set = 6 t of real functions / : S — >• R 1 such that 
E[\f{X)\t] < oo. 

For decreasing function h : Rh — > R\_ and t > 2 dehne the set O = 02, h of real 
functions / such that 

P(\f(X)\ >s" 1 ) <h(s), s>0 (2.16) 

and 

E[f{X)] < co. (2.17) 
All results are given bellow in terms of re-topology. 

Theorem 2.5. Let a sequence a n > 0, a n — > 0, a n+ i/a n — > 1, k n a 2 n — > 00 as n — > 00 
be given. Let 

00 

^2 h{ca n ) < 00 (2.18) 

71=1 

for any c > 0. 

Let Oo C Aoe, h ■ Then there hold 

liminf(fc„a2)- 1 ln(P„)*(P A r e P n + a„O ) > -pg(int(ft ),P) a.*.. (2.19) 

n— ^ 00 

limsup(fc n a I 2 l )- 1 ln(P„)*(-Pfc„ € P» + ^0) < -pg(cI(n ),P) a. s*. (2.20) 



n— >-oo 

and 
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where the closure and the interior of the set Oo in \2.19\) and i2.20\) are considered 
with respect to t@ 2 h -topology. The outer probability measure (P n )* and the inner 
probability measure (P„)* are considered with respect to o~q 2 h -algebra. 

Let O C A o0t ,t > 2 and let a n = o^ 1 /*). Then and are valid 

with the closure and the interior of the set Oo with respect to T@ t -topology. The outer 
probability measure (P n )* and the inner probability measure (P n )* are considered 
with respect to o~Q t -algebra. 

Example. Let S[exp{c|/(Ad)| 7 }] < oo with 7 > for all / € Q. Then we have the 
following asymptotics 

b n = o(n~~^j , d n = o fn _3 =^ 

and 

a n = o (I logn|" 7 ) . 

Thus the conditional MDP for empirical bootstrap measure holds for the wider 
zone than the usual MDP for the empirical measure. 

Theorem 12.61 given below gives rates of convergence in the conditional MDP. 

Theorem 2.6. Let a sequence a n > 0, a n — > 0, a n+ i/a n — > 1, k n a\ — > 00 as n — > 00 
be given. Let function h : R\_ — > Ri_ be such that 

lim nh (—) = (2.21) 

n— >oo V C / 

for any c > 0. 

Let Oo C Aoe t k ,t>2. Then for any e > and n > rio( e , {ki} ( ^L 1 , Oo) there hold 
(Kal)- 1 log(P n )*(P fe * G Pn + a n n Q ) > -p2(inte Mi (O ),-P) - e (2.22) 
and, j//5g(c[e t h (Oo), P) < 00 additionally, 

(fc„<4)- x log(P n )*(-P fc *„ e Pn + a n Q ) < -pl(d(n ),P) + e (2.23) 

on the sets of events having the inner probabilities more than K n = K„(e, Oo) = 
1 - C(e,O )[/3i„ + fan] where fa n = »fe( £Cl ° e " no) ) and fa n = C 2 (e, O )n 1_ *. 
// pl(d(n ), P) = 00, for any L>0 

{Kalr 1 log(P„)*(P fe * n e P n + a„O ) < —L (2.24) 

on the sets of events having the inner probabilities more than K\ n = K\ n {L, Oo) = 
1 - C(L, O )[/3i„ + fa n ] where fa n = ^( Cl (l" 0o) ) and f3 2n = C 2 (L, O )n 1_t . 

Remark 2.2. The method of the proof of Theorem 12.61 is the following. Let / 6 
Q t ,h- Denote ¥j = \f(Xi)\,l < i < n and let < y( 2 ) < . . . < be the order 
statistics of Y 1 ,...,Y n . Using P(maxi<i<„ f{Xi)\ < Ci(Q )ea~ 1 ) > l-nfc( Ci °^ o)e ) 
we, with probability 1 — fa n , prove conditional if < Ci(Qo)ea~ 1 , 1 < i < n 

holds. The rate function in the conditional MDP is defined by ^ X^=i f 2 {X s )- 
Thus we need to estimate the rate of convergence of / 2 (A"s) to Ef 2 (X). 

These estimates causes the second term fan m K n- 

3. Moderate deviation probabilities of statistical functionals 

In section we compare the asymptotics of moderate deviation probabilities of 
statistics T(P n ) - T{P n ) and T(P*) - T(P„). We suppose that the functional 
T : A — > R 1 has the Hadamard derivative or homogeneous. 
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(3-3) 



3.1. Functionals having the Hadamard derivatives. For all r > define the 
set T 0r = {G : p 2 Q (G : P) < r, G G A }. 

Let Y be a metric linear topological space with metric p. We say that the 
functional T : Aos — > Y has the Hadamard derivative T' : Aos — > Y if the following 
assumption Cs holds with S = ^f, £ = $ or £ = 0. 

Cs. For any r > for each G G To r and any sequence Gk € T$ r converging to 
G in T£-topology there holds 

]im8wpp(u^ 1 (T(P +u k G k )-T(P ))-T'(G),0) = (3.1) 
for all sequences u k — > as k — > oo and u& 7^ 0, 1 < fc < 00. 

Theorem 3.1. Assume A, B2,C^ . Let the functional T(P) be continuous in r$- 
topology. Then, for any set S!c7, there holds 

liminf(n^)- 1 lnP„(T(P„) - T(P„) G 6„Q) 

n->oo (3_ 2 ) 
> - inf{pg(G : P ) : T'(G) £ int(ft), G e A } 

and 

limsup(n^)-i lnP„(T(P„) - T(P„) G 6„0) 

n— >-oo 

< -inf{p2( G : P ) : T'(G) G ct(fi),G G A }. 
IfT'(G) is continuous in r^-topology, then, for any 8 > 
limsup^r 1 ln(P„)*(p(^(T(P„) - T(P n ) - T'(P n - P o )),0) > 5) = -00 (3.4) 

The Hadamard differentiability of statistical functionals in the Kolmogorov met- 
ric (supnorm of difference of distribution functions) are the standard tool for the 
proofs of asymptotic normality (see van der Vaart and Wellner [27] Ch 3.9 and 
references therein). Gao and Zhao |16| extended this approach on the moder- 
ate deviation zone. The Kolmogorov metrics is continuous w.r.t. r^-topology (see 
Groeneboom, Oosterhoff and Ruymgaart [T7] Lemma 2.1 and Ermakov [T5] Lemma 
4.1). If S = [a, b] C P 1 , by contraction principle (see Theorem 4.2.1 in Dembo and 
Zeitouni [10]) this implies that MDP holds for the set of empirical distribution 
functions lying in the Banach space of all right continuous with left-hand limits 
functions z : [a,b] —> R 1 equipped with the uniform norm. T Thus Theorem 3.1 
in [TB] can be replaced with Theorem 13.11 of this paper for the study of moderate 
deviations of estimators. At the same time Theorems 13.21 and 13.31 allow to get the 
results for the bootstrap setup as well. 

Theorem 3.2. Assume A, P1,G$. Let the functional T{P) be continuous in r$- 
topology. Then, for any set C Y X Y , there holds 



liminf^r 1 \nP n {(T(P*) - T(P n ),T(P n ) - T(P„) G b n fl) 

n—too 

> -inf{/4(G : P ) : (T'(G 2 ),T'(G 1 )) G int(ft),G 2 x G x G A 2 } 



and 



limsup(n^)- 1 lnP„((T(P„*) - T{P n ),T(P n ) - T(P n ) G & n f2) 

< -inf{/4(G : P ) : (T \G 2 ),T '(Gi)) G ct(fi),G 2 x G x G A 2 Q } 
If ' T' '(G) is continuous in r$ -topology, then, for any S > 

hmsup^)- 1 ln(P„)* (p(d- 1 (T(P*) - T(P„) - T'{P* - P„),0) > 5, 

p{V n {T{P n ) - T(P„) - T'(P n - P )), 0) > 5) = -co 



(3.5) 



(3.6) 



(3.7) 
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Theorem 3.3. Let a sequence a n > 0, a n — > 0, a n +i/a n — > 1, k ri a 2 n — > oo as n — > oo 
oe given. Let a decreasing function h : R\ —> R\_ satisfy \2.18\) . Let the functional 
T(P) be continuous in tq 2 h -topology and let C@ 2 h be valid. 
Then, for any set CI C Y, there holds 

liminf(fc„a2)- 1 ln(P„),(T(P fc * ) - T(P n ) € b n fl) 



> -M{pl(G : P Q ) : T'(G) £ int(0),G £ A } a. fl- 
an d 

limsup(fc, 1 a2)- 1 ln(P n )*(T(P* ) - T(P„) € &„fi) 



(3.8) 



(3.9) 



< - inf{^(G : P ) : T'(G) € cl(fi), G € A } a. s*. 
IfT'(G) is continuous in tq 2H -topology, then, for any 5 > 0. 
]imsuv{k n al)- 1 ]n(P n )*(j)[b-\T(P*) -T(P n ) - T'(P* - P n ),0) > 8) = -oo a.s*. 

(3.10) 

Lei t/ie functional T(P) be continuous in TQ t -topology, t > 2 and /ei Ge t oe valid. 
LetClcY and let a n = o^ 1 / 1 ). Then (31$ . and iTO|) /io/d. IfT'(G) is continu- 
ous in Te t -topology, then i3.10\) is valid as well. 

Theorem 3.4. Let a sequence a„ > 0, a„ — > 0, a n+ i/a n — > 1, fc n a 2 — > oo as n — > oo 
6e given. Let a decreasing function h : R\_ — > R\ satisfy \2.21\) . Let the functional 
T(P) be continuous in re t h -topology and let Ge f h be valid with t > 2. 

Let fl C y . Then, for any e > and n > ?io(e, ^> ^ er e /io?d 

Hminf(fe n 4)- 1 In(P n ).(T(P^ B ) - T(P„) e Ml) 
> - inf{/og(G : P ) : T'(G) G int(fi), G £ A} - e 
and, ifM{pl(G : P Q ) : T'(G) £ cl(Q),G £ A } < oo additionally, 

limsup(fc„c4) - inf{p§(G : P ) : T'(G) £ cl(fi), G £ A } + e (3 . 12) 

n— >oo 

on the sets of events having the inner probabilities more than K n — K n (e,Q,T) = 
1 - C{e,VL)[p ln + (3 2n ] where /3 ln = nh( tCi ^ nT) ) and (i 2n = G 2 (e, fl, T)n 1_t . 

IfT'(G) is continuous in tq -topology, then, for any S > and any L > f/iere 
exists no = uq(L, 6, {ki}'^L 1 , T) such that for all n > uq 

limsup(fc„a^)- 1 ln(P n )*(p(6; 1 (r(P*)-T(P n )-T / (P*-P n ),0) > 5) < -L (3.13) 

n— >oo 

on the sets of events having the inner probabilities more than R n — K n (L,5) = 
1 - C{L,8)[f3 ln + /3 2 «J wnere /3i» = nh ( Cl( ^ T ) ) and p 2n = C 2 {L,5,T)n 1 ~ t . 

For the proof of Theorem 13.11 it suffices to implement the contraction principle 
of Theorem 4.2.23 in [TO] to the sequence of functions f k {G) = b~\T{P n + b k G) - 
T(P n )). In Theorem 4.2.23 in [TO] it is assumed that 

limsup sup p(f k (G),T'(G)) = 0. (3.14) 

Since T r is compact and sequentially compact set in the r^-topology (see Elchels- 
bacher and Schmock [13] Lemma 2.1) then (|3.14[) follows from (13.11) . 

The proof of (|3.4p is akin to the proof of similar statement (3.4) of Theorem 
3.1 in Gao and Zhao [TO] and Theorem 3.9.4 in van der Vaart and Wellner [27] . 
We consider the mapping (j) k : A e ^ Y xY with <f>k(G) = (f k (G),T'(G)) for all 
G £ A oe - By (02]), ([23]), we get that 0„(P„ - P ) satisfies MDP with the rate 
function 

p 2 (yi,y 2 ) = inf{p§(G : P ) : T'(G) = yi = y 2 } EYxY. 
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Hence, by the classical contraction principle (see Theorem 4.2.1 in Dembo and 
Zcitouni [TO]), we get ([3~3f . 

The reasoning in the proofs of Theorems 13.21 l3~3l and l3~4l are similar. 

3.2. Examples. In what follows, Y = R 1 and we shall suppose that the assump- 
tions of Theorems 13.11 and 13.21 are satisfied in the case of moderate deviations of 
empirical measure and bootstrap measure respectively. In this case Cg can be 
rewritten in the following form. 

Cls There exists h : S — > R 1 , E[h(Xi)] = such that, for any r > for each 
G £ Tor and any sequence Gk S To converging to G in r^-topology there holds 

lim u^(T(P + u k G k ) - T(P )) - / hdG = (3.15) 

k— >oo J 

for all sequences Uk — > as k — > and u k ^ 0, 1 < fc < oo. 
By Theorem 13.21 we get 

lim (nbl)- 1 logP n (T(P*) - T(P n ) > b n ) 

n—too 

= lim (nbly 1 logP„ ( fhd(P* - P n ) > b, 

Iff f (3.16) 

= --mlU{gt+gl)dP: J g 2 hdP>l, 9 i,g 2 eL 2 (P) 



h 2 dP 



and, by Theorem 13. 1[ 

lim (ndl)- 1 log P„(T(P„) - T{P n ) > d n ) 

n—too 

= lim (nbl)- 1 logP„ f / hd(P n - P„) > d n 



(3.17) 



= -^inf |y.9 2 dP: J ghdP > 1, g e L 2 (P)} = -^J h 2 dP^ 

By Theorem 13.31 we get 

liminf(fc na 2)- 1 log(P„)*(T(P fe * ) - T(P n ) > o») 

n— J-oo 

= limsup(fc„a, 2 1 )- 1 log(P n )*(T(P ft *J - T(P n ) > a„) = -\ ( [ h 2 dp) a.s. 

(3.18) 

Thus, the asymptotics of moderate deviations probabilities of T(P*) — T(P n ) and 
T{P n ) — T(P n ) coincide. At the same time 

lim (nbl)- 1 logP„(T(P I t) - T(P„) > b n ) 

n—too 

= lim (nbl)- 1 logPn f / hd(P* - P n ) > b n ) 

= 1 inf | y (,g 2 2 + .g 2 ) dP : | (,g 2 - 9l )hdP > 1, fll)fla e L 2 (P)| 
1 " 
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3.3. Homogeneous functionals. Let N : Ao$ — > R 1 be a seminorm continuous 
in the r$-topology. Define the set ilo = {G : N(G) > 1,G E Ao$} and let the 
signed measure H e d(f2o) be such that Pq(H : P) = \ J h 2 dP = Pq(^o : P) with 



h = jjj; . Then we have 



lim (ndl)- 1 \ogP(N(P n -P)> d n ) = -p 2 (n Q : P) = -I f h 2 dP, (3.20) 

n— >oc 2 J 

lim (nbl)- 1 logP(N(P* - P n ) > bn) = 

n—>oo 

-\ml^J{gl+gl)dP : N(G 2 ) > l; 9l = ^ 7 g 2 = G 2 ,G, € A $| = 
-pl(Q :P) = -l J h 2 dP 

(3.21) 

and 

lim (nblr 1 lo S P(N(P: - P) > b n ) = -^(fi : P) = -\ I h 2 dP (3.22) 

n— too Z 4 J 

In particular, the statements (|3.20p and (|3.22p are valid for the functionals 

N(Q -P) = N X {Q -P,P) = max{|F(:r) - F Q (x)\q(F (x)) : x € S} (3.23) 

and 

N(Q -P)= N 2 (Q -P,P)=( [ (F(x) - F (x)) 2 q(F (x))dF (x)) ' (3.24) 



respectively. Here q is a bounded weight function, S — R 1 and F, Fq are the distri- 
bution functions of Q, P respectively. Uq=l, Ni(P n — P,P) and 7V|(P„ — P, P) are 
Kolmogorov and omega-squared test statistics respectively. The functionals N\ , N 2 
depend on the probability measure P additionally. Thus (|3.2ip holds only in the 
case of q = 1. Let us show that, if q is continuous in [0, 1] the presence of weight 
function q does not influence seriously on the asymptotic (|3.2ip . that is, 

lim (nbly 1 logP{Ni(P* - P n , P„) > bn) = 

lkn {nbl)- 1 log PfA^P* - P n ,P) > b n ) = (3 25) 

- P 2 (n Q : p) = - l - J h 2 dP 

with i = l,2. Note that, if q is continuous in [0,1], the following assumption holds. 
CI. There exists function tu(t), u>(t)/t — > as t — > such that, for all P, Q, R E A<j> 
\N(Q - P,P) - N(Q - P, R)\ < w(sup \F(x) - F (x)\) 

X 

where F stands for the distribution function of R. 

Let F n be the distribution function of P n . Then, by Theorem 12.31 
P(w(sup \F n (x) - F (x)\) > cb n ) < cxp{-CnC n b 2 n } 

X 

where C n -> oo as n oo. 

Hence, estimating similarly to the proof of (3.17) in [T5] we get (|3.25p . 

Let us find the asymptotic 



lim (nb 2 ,)- 1 logP(N~<(P: - P) - N~<(P n - P) > b n ) = -\j 



with 7 > 0. 



12 



MIKHAIL ERMAKOV 



By Theorem 12. 1[ we get 

,/ = inf jy (r 2 + g 2 ) dP : N^(G + R) - N~<(G) > 1; 

■ 9 = % ' r = S 5 G ' R E A °* } = inf V{G > R) • 
Since iV(G + P) < AT(G) + iV(P), we get 

J > inf | / (r 2 + 5 2 ) dP : (JV(G) + iV(P)) 7 - 7V 7 (G) > 1; 



(3.26) 



9 = IP' r = IP'' G ' fl G A °*} " inf C/(G ' (3 - 27) 
It is easy to see that for the fixed G 

arg inf U(G, R) = APT, X = A(G) > (3.28) 

where the signed measure H <E cl(Oo) is the same as in example 3.3. 

Let r = Aft. be fixed and let us consider the problem of minimization of U(G, XH) 
with respect to G. We begin with the dual problem. Let N(R) = d =const and 
one needs to find 

sup{(JV(G)+d) 7 -iV 7 (G) : J g 2 dP = 1, g = ^, G e A A 

Let 7 > 1. Since the function (x + d) 1 — x 1 is increasing the supremum is attained 
on the charge G = cG where G = argsup{iV(G) : / g 2 dP = 1, g = G £ A $} 
and g — jfs — h/po- Therefore inf{J7(G, R) : G,R € Ao$} is attained on the 
signed measures G, R having the densities g = ah, r — dh with a,d € R 1 . However 
V(aH, dH) = U{aH, dH). Hence we get 

J = inf{rf 2 + a 2 :{d + a)'' - a< > 1} / h 2 {s) dP. (3.29) 

If 7 < 1, then argsup{(.T + d) 7 - a; 7 : x > 0} = 0. Therefore inf{C/(G, P) : G, P € 
Ao$} = cP and 

J = inf | y r 2 dP:iV 7 (i?)>l,r=^|=2p 2 (r2o,P)= | ft 2 dP. 



4. Proof of Theorem 12.11 
For each r > define the set Y r = {G e A 2 : p 2 h (G : P) < r}. 
Lemma 4.1. Lei PT7] ) ftofti Tften 

i. r r c Aq^ , 

ii. the set T r is Tq> -compact and sequentially -compact set in A 2 ,^, 
m. ifte r and r^, - topologies coincide in T r . 

Proof. The reasoning are akin to the proof of Lemma 2.1 in Eichelsbacher and 
Schmock (T3]. For any charge G = G\ X Gi € T r , any measurable set ACS and 
any 4>i,4>2 G ^ we have 



/ \c/> 1 \d\G 1 \+ [ \<h\d\G 2 \ < 

J A J A 



(dGi\ 2 f (dG 2 ^ 2 



A 



%dP+ I tidP)+a-' \ I dP+ l^p-) dP 



for all a > 0. By (pTTUj) . this implies Hi A = 3. 
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Fix e > 0. Let a = r/e and let n = n(e) be such that 



<t>ldP+ 4>ldP] < e 

\<j>i\>n J\<t>2\>n J 



Then 

-iff fdGA 2 JD , f fdG 2 K 2 



a 



\</>i\>n \ dP J J\<p2\>n V dP 



dP+ — f dP\ < 



Hence, by (|4.ip . we get 

f \ct> x \d\G x \+ f\fa\d\G 2 \- f \4> x \d\G x \- f \cb 2 \d\G 2 \<2e 

J J J\4>i\<n J\<t>2\<n 

Therefore the map r r 9 G = G\ x G2 J \<f) X \ d\G x \ + J \(f> 2 \ d\G 2 \ is r-continuous 
as the uniform limit of functions 



<f> 1 dG x + / cp 2 dG 2 . 

\<t>i\<n J\4>2\<n 

This implies that the r and 7^,-topologies coincide in T r . Since the sets Fo r and 
T r C Tq t are r-compact and sequentially r-compact these sets are T$-compact and 
sequentially r^-compact as well. This completes the proof of Lemma |4. II 



The same reasoning of the proof of Lemma 14.11 can be repeated in the case of 
r$-topology. Thus the sets To r are r$- compact as well. 

In Lemmas 4.2-4.5 given bellow we suppose that the assumptions of Theorem 
Ohold. 

For any u,v G R k denote u'v the inner product of u and v. For any / G $ and 
any signed measure G G A $ denote < /, G >= J fdG. 

Let f x ,...J kl , gi ,...,g k2 G $ and G G A $. Let E[fi(X)] = 0,E[ 9j (X)] = 
with I < i < k x ,l < j < k 2 . Define the covariance matrices Rf = {E[fi(X) fj(X)]} kl j =1 
and R g = {E[ gi (X)g 3 (X)}}^ =1 . Denote / = {/^ , g = {g^l, and g { = 

iYZ=i9i{Xi)A<i<k2. 

By a version of Dawson-Gartner Theorem (see [TU] Theorem 4.6.9 and |21|). 
Theorem 12.11 follows from Lemma 14.21 given below. Note that the de Acosta [5] 
approach (see section 5) also allows to deduce Theorem 12.11 from Lemma l4~2l 

Lemma 4.2. For the random vectors U n (X) = (i Y*=i h( x i)> ■ ■ ■ >n S"=i / fc i > 
TlT l ti9i(X*)-g x ,---, : kT l ti9k 2 (X*)~g k2 ) the MDP holds, that is, for any 

n c R k i+ k * 

limmiinbl)- 1 log P n {U n {X) G b n Q) > - inf x'I fg x (4.2) 
™- >0 ° x£int{a.) 

and 

limsup(n6 2 l ) _1 logP n (f/„(X) G b n Cl) < — inf x'I f , g x (4.3) 

n->oc xecl(n) 

where for any x = (y, z) G R kl+k2 , y G R kl and z G R k2 

x'If jg x= sup (t'y + s'z— <t'f,H > —-t'Rft — -s'R g s 

t£R k l ,s£R k 2 \ 2 2 

Note that, if there exist RJ 1 and R~ l , then 

x'Ifgx = \{{y~ < f,H>)'Rf(y- <f,H>) + \z'R- x z. 
Lemma 14.21 follows from Lemmas 14.31 and 14.41 given below. 
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Lemma 4.3. We have 



and 



lim (no„) logP„( max max \fi(Xi)\ > 6„ ) = -co (4.4) 

n— >oo l<i<fcx l<Z<ri 



lim (nb 2 n ) x logP„( max max \gi(X*)\ > b,^) = -co. (4.5) 

n— >oo KKfa KKn 



Proof. We have 

fci 



P„( max max > b~ l ) < P„(|/ i (X 1 )| > < 



i=l 

fei fci /» 

^P(|/ 4 (X!)| >6- 1 )+nb„X; / X (\fi(Xi)\>b- 1 )d\H n 
„- — i a — i 



i=l i=l 



By (|2~Tj) and Bl, this implies (j4~4l . 

Since gx, . . . ,gt 2 G the same statement hold for these functions as well and 
we get 

p n( max max > o" 1 ) = 0(exp{-Cn&£}) 

1<2</C2 l<J< n 

for each C > 0. This implies (|4~5j) . 

For any ft£$ denote /i„(a;) = h(a;)x(|h(aO| < b~ x ). Denote /„ = {fi n }iLi an d 
9n = {9in}i=v Define the random vector U n (X) = (i £" =1 fi n (Xi), . . . , ± Yh=i fk 1 n{X l ), 
7lTZ=i9in(X?)-gi n ,..., ^Y,i=i 9k 2 n{X?) -g k2 n) where^ n = i £)/=i 9in{Xi), 1 < 
i < fc2- Define the events W n = {X±,...,X n : maxi<j<fe 1 maxx<3<n |/t(-^i)| < 
b" 1 , maxi<i<fc 2 maxi<j<„ < b~ 1 }. Denote W„ the complement of the event 

W n . 

By Lemma 4.2, we get 

Pn(Un(X) G 6„0) < P n (Un(X) G 6 n O|W„)P(W n ) + P(W n ) 
< P n {U n (X) G &„tt|W„)cxp{o(n&2)} + e xp{-Cn&£(l + o(l))} 

and 

P n (Un(X) G 6„0) > P n 0n{X) G 6 n Q| VK„)P(M>„) > P n {U n {X) G WWn) exp{o(n&2 )}. 

(4.7) 

where the constant C in (|4.6[) can be chosen arbitrary 

Therefore Lemma 4.2 follows from Lemma 4.4 given below. 

Lemma 4.4. Por £/ie random vectors U n (X) the MDP holds, that is, J^.g| j and 
ll^D are va&d mift P„(J?) = P„(X). 

By Gartner-Ellis Theorem (see Dembo and Zeitouni [TU]) Lemma 14.41 follows 
from Lemma 14.51 given below. 

Lemma 4.5. Let f, G <£>, ffj £ $ /or all 1 < i < fci, 1 < j < fc 2 . T/ien 



lim (no 2 ) 1 logP„ 

n— >oo 



exp J 6„ t'MXi) +bnJ2 s '(9n{Xt) 



k 1=1 1=1 
<t'f,H>-U l R } t-^s'R gS 

where g n = (g~i n , . . . ,§k 2 n)- 



(4.8) 
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Proof. We begin with the proof of upper bound in (|4.8|) , We have 

{n n *\~ 

1=1 1=1 ) . 

{n \ n 

1=1 J 1=1 

E n exp \ b n y~] t'f n (Xi) I ( - y~] exp{b n s'(g n (Xi) - g n )} ) 



< 



E,, 



CXp \ b n J2 *fn{Xl) \( 1 + ^ J2( S '(UXl) ~ gn)) 



1 = 1 



hi 



1=1 
" 1 



+C(s,k 2 )-^^ s '(9n(Xi) - g n )\ 3 



i=i 



< 



n 



exp b n t'fn(Xl) + f J2(s'(9n(Xl) ~ 9n)f 



1 = 1 



C(s,k 2 )b 3 n ^\s'(g n (X l )-g, 



1=1 

M3 I 



1 = 1 



The first inequality in (|4.9|) follows from the Taylor formula and 
\s'(gl(x) -g n )\< \s\ \gn(x) - g n \ < l^fc^V 



% 2 u n 

Denote <t> n (Xi) = s'(g n (Xi) - E n [g n (Xi)}) with 1 < I < n. 
By straightforward calculations, we get 

n n 

£>'(s„(X ; ) - g n )f =J2€(Xi) ~ n(s'g n - E n [s' g n (Xi)]f 
1=1 1=1 

We have 



^'{UX^-gn)? <%Y,\^ X ^' 



1=1 



1=1 



Since 



8n\s'(g n - E n [g n {X 1 )))\ 3 = Wi + 8nV 2 . 
s'{g n {Xi) - EgniX^l 3 < {s^lg^X,) - E^niX^ 2 

3/2 



k 2 



kl 372 ( £G&n(*l) ~ En^niX,)) 2 ] < 8 1 S | 
J =1 



we get 



bl\Vi\ =b 3 n J2 \MXi)\ 3 x(\MXi)\ < eb- 1 \s\)+ 

1=1 

n 
1=1 

n n 



(4.9) 



(4.10) 



(4.11) 



(4.12) 



(4.13) 



(4.14) 



i=i 



1=1 
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By the Jensen's inequality, we get 



V 2 = n~ 3 



By (HTT1) - (HTT5l) . we get 



1=1 



^n'^lMXi)] 3 =n- 1 Vi. 



(4.15) 



i=i 



In 



exp i b n £ t'f n {X l ) + -f (1 - 2<7(s, fc 2 )e„) ^(^(X,) 



;=i 

\ 2 



/ = 1 



X>„(*0 +c( s ,fc 2 )| s | 3 ]Tx(l<M^)l > cft-^l)) 



^/ = l 



i=l 



= E n [W n ] 

(4.16) 

where e = e n — >0asn— >oo. 

For each r > define the events ^4„ = A„ r = {Xi, . . . , X n : s'g n — E n [s' g n (X\)] < 
rb n } . Denote A n the complement of A n . 

We can write 



I n = E n [W nX (A n )] + E n [W nX (A n )} = U ln + U 2n . 
Let A n hold. Then we get 

/ _ v 2 

rib 



(4.17) 



2rc 



2 = -^( S '5n " i?n[ S 'ffn(X)]) 2 < 



U=l 



Therefore 
log[Cln] <l0g^„ 



exp J &„ £ i'&pC,) + f £ ^(X,)(l + 2C(s, fc 2 )e)- 



i=i 



C( S ,fc 2 )| S | 3 ^ X (|^(X ; )|>e6- 1 ) + 0(nr 2 ^) 



i=i 



(4.18) 



n log 



M7n(*i) + y^(X0(l + 2C(s, fc 2 )e)+ 



C( S ,fc 2 )| S | 3 x(|^ l (X 1 )|>e6^ 1 ) + 0(r 2 64)}] 
Expanding in the Taylor series, we get 



logf/i n < nlog£;„ 



l + U7n(ll) + y(i7n(^l)) 2 + 



« J.2 



(4.19) 



where u> n = u ln + uj 2n + co 3n + uj An + us 5n with 

Win = |-|i7n(*l)| 3 , L0 2n = 3^|i7n(^l)|^„(^l), 

W3 „ = f ^(Xr), W4 „ = ^(t7n(*i)) 2 <^(Xi), 
co 5n = x {\(f>n(Xi)\>eb- 1 ). 

We have 

< 6 3 J*7n(*i)| 3 x(|i7»(Xi)| < e^ 1 )+ X (e6^ 1 < |*7»(*i)l < O = Wi„i+w ln2 , 

W 2 „ < 6 3 |t7n(^l)l^(Xl)x(|t7n(Xl)l < ZK X ) + 

C*(s,i,fci,fc 2 )x(e6^ 1 < |t'.^(^i)| < b- 1 ) = u 2nl +uj 2n2 , 
w 3 „ < b A n 4> A n {Xi) X {K{Xx) < eb- 1 ) + C X (eb- 1 < 4> n (X x ) < c^ 1 ) — W3„i + W3„ 2 , 
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u 4n < bt l (i/f n (X 1 )) 2 <p 2 n (X 1 )x(\^fn(X 1 )\ < e6.- 1 )+ 

Cx(e6~ 1 < \t'f n (Xi)\ < K 1 ) = ^inl +U 4n2 . 

By (EHJ, we get 

E n [u lnl ] < ce\t\b 2 n E n {t' f^)) 2 , E n [u 2nl ] < ce\t\b 2 n E n <l> 2 n {Xi), 
E n [LJ 3nl ) < ce^sfblE^KXi), E n [u> 4nl ) < ce^blE^UX^ 

and 

E n [uj 5n ] < e- 2 b 2 n E n [4>l{X 1 ) X {\MX l )\ > eb- 1 )] = o(e"X), (4-20) 
E n \x{eb~ l < \tf n {X{)\ <b- 1 )} < 

e^blE^t'UX^xieb- 1 < \t' fn(Xi)\)] = o{<T 2 b 2 n ) 
where the last equalities in Qjj2DJ) , (ji^l]) hold by A and flU), pTTU]) . 
Hence we get i? n [u; n ] = 0(6^). 
Therefore we get 

jib 2 f -* \ 
log(U ln ) < ^ (2 < t'f, H > -t'Rft - s'Rgs) (1 + 0(1)) = v n . (4.22) 



(4.21) 



2 

By the Hoclder's inequality, we get 

U 2n < [E n [W^ +s ])^(P(A n ))rU. 

By (j4~TB|) . we get 



E n [W 1+s ] < E 



exp \(l + S)(b n J2 t'UXi) + b 2 n J2 &(Xi)Q- + 2C(s, fc 2 )e)- 



2C( S ,k 2 )J2x(MXi)>eb~ 1 )j 

Hence, repeating the estimates of U\ n , we get 

E n [W 1+s ] < exp |- (1 + S 2 )nb " (2 < t'f, H > -t'R f t - s'R a s)(l + 0(1))} (4.24) 

Note that (j2TTj) implies (|2~7| and ([2~7)l implies 

lim (nr 2 ^)" 1 iogfa-PG/PQI > rnb n )) = -00 

n— ^00 

for each r > 1. 

Hence, by Theorem 2.4 in Arcones [2], we get 

logP„(A„)<-cr 2 n&2 (4.25) 
By (|433)) . (j4T24)) . (|435|) we get that 

U 2n = o(U ln ) (4.26) 

if r sufficiently large. This completes the proof of upper bound for /„. 

The proof of lower bound is based on similar estimates. Define the events 

B n = {xi, ...,x n : \f n i(x s )\ < eb' 1 , \g nj (x s )\ < eb' 1 , 1 < s < n, 1 < i < ki, 1 < j < k 2 }. 

By (EU), (12201), dMD, we get 

P n (\fm{Xi)\ > eb' 1 ) < e-XEnlfii(Xi)x(\f n i(Xi)\ > tK 1 )] = o(e~ 2 b 2 n ). 

Estimating P n (9ni(^i)\ > e^" 1 ) similarly, wc get 

All &2 

P{B n ) = n(l-P(|/ ni (Xi)| > eb- 1 ))" n(l-P(| 5 ni(Xx)| > eb- 1 ))" = cxp{- (n^)}. 
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Hence 



exp { b n t'f n (Xj) | [ i ^cxp{6„s'((?„(X i ) - g„)} J x(5„) 



i=i 



i=i 



< 





exp < 


\b n f^t' f n {Xi) 

I i=l 


K 




exp j 


bnf^ffliXiV 
i=l J 


■( 



= 7 2n ex P{-°("^)}- 
Expanding in the Taylor series, we get 



exp{-o(n&^)} 
(4.27) 



lln. > En 



h 2 n 



i=l 



h3 n 

C( S) M-5>'(M)-Sn)| S 



i=l 



> 



exp <j 6 n £>7«PQ) M 1 + 2^(1 " 2e) j>'(&(*i) - 9n))' 



— 

(4.28) 



where the last inequality follows from 

n n 

J2 WWnfr) - g n )\ 3 < 2eb^ J2(s'(9n(Xi) ~ 9n)f 
t=l t=l 

Since ln(l + x) > 1 + x — x 2 with x > we get 



'3n 



n 



exp I b n J2 t'fn(Xi) \explnhx[l+ f(l- 2e) ^( S '(.g„(X 4 ) - g„)) 2 



exp \ b n J2 t'MX) + -f (1 - 2e) £>'(£„pQ) - g„)) 2 



^ (^ (s '(3„ ( X0-5n)) 5 



B„ 



> 



exv\b n J2t'f n (Xi)+ 



h 2 " 

^(l-2 £ -4 £ 2 )^(.s'(.g„(X0-g„)) 5 

i=l 



where the last inequality follows from 



(4.29) 



4» 



I 5>'(&(Xi) -s„)) a < e 2 blY J (A9n{x l )-9n)f 
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Arguing similarly to the proof of upper bound we get 

(nbl)- 1 hxh n = (-2 < t'f, H > -t'Rft - (1 - 2e - 2e 2 )s> ' R gS ) (1 + 0(1)). 



(4.30) 

Since the choice of e > is arbitrary, this completes the proof of lower bound and 
the proof of Lemma 14.51 

5. Proofs of Theorems 12.51 and 12.61 



We begin with the proof of Theorem 12.61 The reasoning is akin to the proof of 
the Sanov Theorem in de Acosta |S]. 

Lemma 5.1. . Let (STfy - [M5\) hold. Then 

i. r 0r C A e, 

ii. the set Tor is TQ-compact and sequentially TQ-compact set in Aqq, 
Hi. the t and tq - topologies coincide in To r . 



The proof of Lemma [5.11 is akin to the proof of Lemma [4.11 and is omitted. It 
suffices to note only that (|2.17[) holds. 

We begin with the proof of upper bound in (|2.23[) . Denote rj = /?o(cl(Oo), P) and 
fix S, < 2(5 < T). IT is clear that T , v s C A e \ 

For any /i, ...,/;€ 6, G € Ao,e and 7 > denote 



J fid(R - G) 



< j,R £ A e,l <i<l\ . 



Define the linear space 

k 

A oe = {G : G = ^ \G l ,G l € A oe , A, e R\ 1 < i < k, k = 1, 2, . . .}. 

i=l 

Define tq- topology in A e- It is clear that A e C A e- 

Since Aoe is the Hausdorff linear topological space, the space Aoe is regular 
space (see Theorem B2 in [IH]). Thus for each G € Tq^-s there exists open set 
U(fi, ...,/{, G, 7) C Aoe \ cl(Qo)- The set ro. v -s is compact. Therefore there 
exists finite covering of To,,,-*- by the sets U\ = U(fn, ■ ■ , fui>G~L> c i)> ■ ■ ■ , U m = 
U(f m i, fmi m , G m , Cm ) De note U = ^T=i u i- 

Thus for the proof of (|2.23p it suffices to estimate 

P n (P* £P + a n U) > (P n )*(P r : eP + a„0). 

This is the finite dimensional problem. 

For each < j < h,l < i < m define the signed measure Fij having the 

density ^p- = fij — E[fij(X)}. Define the linear spaces 

k h 

L = {F:F = J2J2 X i3 F iJ> X ij e Rl > 1 ^ 3 < U, 1 < * < m}, 

i=i j=i 

and 

Define the sets f 0c ={/:/ = %, F E T Qc n L) , c> 0. 

There exists a finite number of functions q±,...,qi G T v -2$ such that E[qi(X)] = 
0, E[qf(X)} = 2(77 - 25), 1 < i < I and 

r, ) _ 2 «5 n L c d l=1 v{ q .i) nic T v _ s n L (5.1) 
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with 

Vi = V(q t ) = ■ 

Denote 



q%dG 



< 2(ri-2S),Ge A 



OH 



(5.2) 



Since r,, ; _,5 n L C U n L we get V C U. Hence 

CW = A 09 \ V. 
Therefore it suffices to estimate the right hand-side of 

log(P n )*(Pfe„ G P n + a n Q ) < log P„(P fc * n G P„ + OnW). 

We have 

fc 

P n (P£ 6 P„ + a n W) < Pn{ p k n £Pn + a n Ui) = 
i=l 

k 

P n (/ « d ( P fe„ - A.) - 2 <*n(»7 - 2S ) > °J • 

Thus it suffices to show that, for each / e 6, E[f(X)] = 0, £[/ 2 (X)] = rj - 26 and 
n > rio(e, /), 

log P„ (7 /d(P£ - P n ) > 2a„(77- 25)) < -2 ^"^ (l- e ) = ( ? /-2<5)(f-e) 

(5.3) 

with probability K„(e, Z7(/, q)). 

Denote S 2 = s 2 = 4 = I £™ =i ^(jsg _ p with / = I ^ /(Xs ). We put 

7 = ^ where a 2 = Var[/(Xi)] = 77 - 25. 

By Theorem 28 Ch.4 in Petrov [23] we get P(|s 2 - a 2 \ > e) < /3 2n (/) with 
/?2n(/) = Ci(/, e)n 1_ *. Thus, to prove (|5.3p . we can suppose that 

14 -<7 2 |<€. (5.4) 

Define the sets of events A n j = {Xi,...,X n : maxi<,s<„ \ f(X s )\ < aja^ 1 }. We 
have 



P(A nf ) < 1 - nP(\f(X!)\ > <r 7 = 1 - nh (^j = 1 - ft 



(5.5) 



Note that, by (|2.2ip . nh[^\ — > as rn- 00. Therefore it suffices to prove f|5 . 3f) . 
if A n f hold. 

The further reasoning is based on slightly simplified version of Theorem 3.2 in 
[26) . This version of Theorem is given below. 

Let Y ln ,.. -,Yk n ,n be i.i.d.r.v'.'s having pmP„, E[Y ln ] = 0, Var[Yt„] = er 2 , \Y m \ < 
aja~ . Denote 

fen 

9 - V Y 

u n rj — / 1 in- 



, =1 

Suppose that 

a~ 2 z~ 2 logi?[cxp{za„cr _1 yi„}] < C if |z| < k. (5.6) 

and 



^ > 1. (5.7) 



36max{l,C} 



1 /2 

Denote A = uja n k n 
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Theorem 5.1. Assume 15.6\5. 7\ ). Then 

P(S n > kl! 2 a n ) = (1 - $(fcy 2 a„))exp{L(fcy 2 a„)} [ 1 + 6 h{kl' 2 a n ) 



fan Qji H~ 1 



(5.8) 



with 



and 



60(1 + 10A 2 exp{-(l - w-^VA}) 



1 - UJ n 1 



6uj 2 1 + uj 

1 /2 

Note that, if uj > 16 and a„fc„ > 100 then 

iw-yv)i fc " /2fl A " +1 <e. 

If |z| < k and < erfa^ 1 , 1 < i < n, we have 

logiJp |exp{za„(/(X*) - /)/*}} = 



(5.9) 



(5.10) 



log 



I - 

- V exp{za„(/(JQ) - f)/s} 

II — * 



log 1 



i=i 

z'~a; 



(in 



2 - /) 3 eM0za n (f(Xi) - 1)1 '8} = r n 

i=l / 

(5.11) 



with < < 1. 
Since 

exp{0za„(/(X 1 ) - /)/«} < exp{2 7K 0a S - 1 } = fl, 
using ln(l + x) < x, x > 0, we get 

/ 2 2 \ 2 2 

r„ <log( l + ^(l + 7 K ( ri?s- 1 ) J < ^(l+ 7 KCTi?s" 1 ) = z 2 a 2 D (5.12) 



with D = ktUlRZLl 
If 



then R < 3 and D < 2. Therefore 

9 



(5.13) 



,1/2 2 



Hence, by ([5\8 )l . (|5TiO]) . we get 

(fc^ 2 )- 1 logP n ( [ fd(Pt - P n ) > 2a n ( V -25)) < 



-s- 2 ( V -2S) 2 1 



9/2 + e 



+ (log 7 - - log(27r S - 2 (l + e)))(fc„a 2 )- x < 



- -s-\r, - 26)' (1 - - j +C(fcX) _1 
= -^^ 2 ('?-2^ 2 (l-|) + C(A : „a 2 )- 1 
<-^^ 2 (^-2^) 2 (l-0+C(fc„a 2 r 



(5.14) 



22 



MIKHAIL ERMAKOV 



This implies (|5.3|) if (|5.4p and \f(Xi)\ < cr^a n l , 1 < i < n hold. This completes the 
proof of IJ223J. 

If /9o( c K^o)i P) = °Oj we put ?y = L. After that we implement the same reasoning 
for the proof of (f2T24|) . 

The proof of lower bound (|2.22[) is based on standard reasoning (see Sanov [H] > 
Dembo and Zeitouni [TU], de Acosta [5] and references therein) and estimates of 
Theorem 15.11 For any 8 > there exists open set U = U (/i ,...,//, G, 71 , . . . , 7z ) 
such that ?7 C int(Oo) and Pq(U, P) < r) + 5, Po(G, P) < r/ + 5. Hence it suffices to 
find the lower bound for the asymptotic 

(Kal)- 1 logP„(P fc * e Pa + a n U). 
Similarly to the proof of upper bound we can suppose that the signed measure G 
has the density g — jjs — J2\=i ^ifci fi £ ®- Thus the problem became the finite 
dimensional problem as well. 

Let us fix A, < A < 1 such that AG £ U. Note that value of A can be chosen 
arbitrary from some vicinity of 1. Define the set U\ = U fl U(g, G, 2(1 — A) 2 ||<?|| 2 ). 
It is clear that we can choose A so that Pq{U\ ■ P) < A 2 ||(7|| 2 

Lemma 5.2. There exist simplex U C U± bounded the hyperplane II = {R : 
j g dR = 2A 2 ||<7|| 2 , R £ Aoe} and hyperplanes LI^ = {R : J gidR = Ci,R £ Aoe} 
with gi£@,l<i<l such that pg(IL : P) > X 2 \\g\\ 2 = Po( n : p )- 

Let Lemma 15.21 be valid. Suppose Af>f holds with / = g and / = <?j, 1 < i < /. 
Then, applying Theorem 15.11 we get 



P n {Pl n £ P n + OnUl) > Pn{P* kn G Pn + CL n U) > 

P„ (j gd{Pt n - P n ) > 2A 2 || 5 || 2 a „^ - 
- P n ) > a„Ci\ > 



(5.15) 



P n (J ' gd{P* kn - P n ) > 2A 2 || 5 || 2 a„^ - Eexp{-p 2 (n ? : P)a 2 fc„(l + e n )}. 

with e n — > as n — > 00. 

Thus it remains to implement Theorem 15.11 to the first addendum in the right- 
hand side of (|5.15l) . 

By dSHJ and ([Slty . we get 

(a 2 fc„)" 1 logP„ ( j gdP* kn - P n ) > a n \\g\\ \ > 

\ (5-16) 
- \\\g\\ 2 (l + ^) + c{k n a n )-* = -\\\gf (l + ±e) + c(k n al)-\ 

This completes the proof of lower bound. 

Proof of Lemma 1 5. ,21 The problem is reduced to the following. There is given a 
parallelepiped U\ in R l+1 and ^ U\, p(0, U\) = inf^gUi 1^1- The point u lies on the 
face II of parallelepiped U\ and p(0, u) = p(0, U\). One needs to point out simplex 
V C Ui such that II n V is the face of V, u £ II n V and for any hyperplane LIi 
passing through another face of V it holds p(0,IIi) > p(0,u). Let the distance of 
u from any face other than II exceeds rQ. A simple trigonometric reasoning shows 
that the simplex V can be defined as follows. We take the vertex v = (1 + \r 2 )u 
of V where r << rg and all other vertices Vi, 1 < i < I belong II and \vi — u\ = r. 

For the proof of this statement it suffices to consider the case 1 = 1. Let us 
draw through v the line L intersecting the line II at the point w and such that w 
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is orthogonal to L. Then |u — v\ = \w — m| 2 |u| _1 (1 + o(l)). Therefore, if the line 
Li,v G L\ intersect II at the point z = c\w — u\ \u\ ,c < 1 then p(0,L\) > \u\. 
This completes the proof of Lemma 15.21 

Proof of Theorem \2.5i The reasoning is based on estimates of Theorem 12.61 We 
begin with the proof of upper bound (|2.20[) in the case of tq 2Ii -topology. Suppose 
that Pq(cI(£Io), P) < oo. If p§(c[(f2o), P) = oo, the reasoning are similar. It suffices 
to prove that for any e > 

(A;„4)- 1 log(P„)*(P fc * n G P n + a n Q ) < -p 2 (d(fl ),P) + e a.s. (5.17) 

By the Strong Law of Large Numbers and (|2.17[) . for any / £ 8, we get 

a. s. (5.18) 

with cr 2 (/) < oo. 

By (|^T5|) and (|2~T6")l . for any 8 > 0, we get 



PCmaxa.l/pQI < <0 = I^ 1 " > *0) 

oo oo 

> [](1 - /»(<*/<*)) > exp{^ h( ai /S)} = 1 + o(l). 



(5.19) 



i—l i—l 

as Z — > oo. 
For any k, 

P( max a„|/(X0| > 8) = o(l) (5.20) 

l<i<fc 

as n — ^ oo. 

Note that maxj>fc a,i\f(Xi)\ < 8 implies \f(Xi)\ < Sa' 1 . Therefore, 

by HUH) and (530]) . we get 

max I/PQI < (5a,; 1 a.s. (5.21) 

Ks<n 



Using (|5.18p . (|5.2ip . we can implement the same technique for the proof of ((th 
as in the proof of l|2.22jl in Theorem 12.61 This completes the proof of (|2.20l 

For the proof of (|2.20p in the case of re, -topology it suffices to show that, for 
any 8 > 



I k =P{maxa i \f(X i )\>8)=o(l) (5.22) 

i>k 



as k — y oo . 
We have 



h<J2P(f(X l )>8a- 1 ) 

i—k 
oo 

= ]T („ - fc)p(fc- l 1 < i/(x s )i < ^r 1 ) = 



(5.23) 



i=fe+l 



Define the function u{x) = Sa,^ + 8(a i 1 — a-_ 1 1 )(x — i + 1) if x G [a i _ 1 ,a j 1 ). 
Define the inverse function v(y) = inf{t : u(t) — y,t G P 1 }. Define the distribution 
function P(x) = P(|/(X)| < x), x G P^. 
Then 

/OO /"OO 
i v{x)dF(x) <2 i X t dF(x) = o(l) (5.24) 

as fc — > oo. This implies (|5.22[) . 

The proof of lower bound (|2 . 1 9[) is based on similar reasoning and is omitted. 
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6. Appendix 
Proof of Theorem \2.4\ One needs to show that 



- logP V > ne n j = o{ne 2 n ) (6.1) 

Define the events A n i = U n i U V n i,l < i < n with U n i = {Yi : \Yi\ < and 
Vni = {Yi : r n < Yi}. Denote A n = nf =1 A nl . 
By flU}, we get 



P(An) > 1 - P( max \Y i \>b~ 1 )> 

l<i<n 

l-nP(\Y 1 \>b- 1 ) = l + o(l). 

Denote P cn the conditional probability measure Y\ given Y\ <G A n \. 
By we get 

Pen [Y.Y* >ne n ) (l + o(l)). 



(6.2) 



(6.3) 



Thus it suffices to prove (|6.1[) with pm P replaced by pm P cn . Denote p n = 
Pcn{Y\ > r n ). By (|2.2p . we get np n — ► as n . — > oo. Define the events W„(fc n ) = 
{ ii, . . . , Y„ : n — k n random variables Y±, . . . ,Y n belong (0, b~ l ) and k n random 
variables Y±, . . . , Y n belong (r n , oo) }. Suppose that k = k n — > oo as n — > oo and 

lim fc„np„ = 0, lim (r„e„) _1 log — ; — = 0. (6.4) 

By the Stirling formula, we get 

tfc = P cn (W n {k)) = ^^(1- Pn )»-* = 

{2n)- 1/2 exp{(n + 1/2) logn - (n - fc + 1/2) log(n - fc) - (fc + 1/2) logfc+ 
fclogp„ + {n-k) log(l - p n )}(l + o(l)) = 

exp f -(n - k + 1/2) log " ~ * - fclog —(1 + (l))j = (6.5) 
L ra(l-p„) np n J 

exp{-n(l - k/n)(-k/n + p n )(l + o(l)) - fc log[fc/(np„)](l + o(l))} = 

exp{(fc - np„ - fclog(fc/(np„))(l + o(l))} 
r £ 

exp<^ -fclog (l + o(l)) 

L np n 

It follows from (|2.13[) . (|6.5[) that we can choose fc = k n , such that 

| log V „| = 0(k n \\og(n Pn )\) = o{ne 2 n ). (6.6) 
Define the random variable l n which equals the number of Y* , 1 < i < n such that 

2 

Y* G (r n ,oo). Denote u„ = c 1 ^- = c r e " with c > 1 and denote m„ = [u n ]. 
Suppose that ^ — >• oo as n — > oo. Then estimating similarly to (|6.5[) we get 

P c {l n > u n \W n {k n )) = exp (-u n log ^(1 + o(l))j (6.7) 
Denote ci = c- 1. Denote y 1 * < . . . < F™* the order statistics of Yf, . . . , Y*. 
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The event {Y?, ...,Y* : X)"=i Y * > ne n } contains the event 

!n—m n 
Y*,...,Y*: J2 Yi* > - Cl ne n , \Y**\ < b~\ 

1 < j < n — m n , Y** > r n ,n — m n < t < n} 

since, if U n holds, 



Y > r n m n = cr n = cne n . 

r n 

t—n—m n —l 

Hence it suffices to show that 

log P c (U n ) =o(nel). (6.8) 

We have 

(n—m n \ 
Y i > -cme n , \Y* \ < b-\ l<i<n- mj > 

(7i—rn n 
V > -cme n , \Y*\ < b-\ l<i<n- m r 
i=i 

(6.9) 

Denote q n = P c (|Fi| < b~ r ). Define the conditional probability measure P bn of the 
random variable Y\ given | Y\ \ < b~ 1 . 
We have 

— ' In — i 1 17. i rj 



We have 



(6.10) 



(=1 

/ n—m n 



(6.11) 



1 - P c £ 1? < -cme„| < 6" 1 , 1 < i < n -m n 
\ i=i j 
By Chebyshev inequality, using (|6.10[) . we get 

/n—m n \ 

P c [ Y * < -d ne n\ \Y*\ < b-\ 1 < i < n -m n < 



\ i=l 

n — m 



c\{n - m n ) 2 e 



^—E c [N & T P (Y*\\Yn<b- 1 )] 



9 i \9 9 c L v t * A P 



„2 » 

£CX(l-?n) n -%„ 



cj(n - m„)e 



I. 



fl 2 .A t - 1 O 2 



_ gn) n-* Var [y] < 2 V*r bn [Y}. 



cf (n - m n )e£ ^ i cf(n - m n 

(6.12) 

and 

lim g 2 Var 6 „[y] = Var[Y]. (6.13) 
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By ||6.5l6.7p . we get 

PcQn = m n \W n {k n ))P c (W n (k n )) = 

cxp a log — - cfe n log —(1 + o(l)) f = exp{-o(n4)} 

where the last equality follows from (|6.4I6.6[) . Now (|6.8[) follows from (|6. 11116.14")) . 
This completes the proof of Theorem 12.41 
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